Skip to content

prep_wake_samples --clean: clear a re-recorded take's orphan clips (#8) - #85

Merged
jamditis merged 2 commits into
mainfrom
fix/8-clean-stale-clips-on-rerun
Jul 23, 2026
Merged

prep_wake_samples --clean: clear a re-recorded take's orphan clips (#8)#85
jamditis merged 2 commits into
mainfrom
fix/8-clean-stale-clips-on-rerun

Conversation

@jamditis

Copy link
Copy Markdown
Owner

What

Adds a --clean flag to prep_wake_samples.py. Re-recording a take with fewer
utterances and re-running prep into a populated --output dir left the previous
run's higher-numbered clips on disk. Training and eval glob every clip in the
directory, so those orphans were silently folded into the next run's data while
process() reported only the count it had just written. --clean removes them in
the same run so the reported count matches what is on disk.

Why it's safe

Deletion is scoped to the current run's own stems: a <stem>_NNN.wav clip whose
stem this run wrote. A source recording, an unrelated take's clips, or hand-curated
audio in the directory is named in the output but never auto-deleted, so a stray
--output can't wipe files prep did not create. Without --clean the existing
warn-and-name behavior is unchanged (deletion is opt-in).

The stem match is a greedy (.+)_\d{3,}\.wav, so an orphan pairs back to its
disambiguated collider stem (computah-1_002.wav -> computah-1, not computah)
and the >999-segment case (:03d is a minimum width, so index 1000 is
take_1000.wav) still parses.

Tests

test_prep_wake_samples.py gains coverage for the re-run-with-fewer-clips case
with and without --clean, that --clean spares files this run did not record,
and disambiguated-stem orphan cleanup. Full suite: 31/31 checks, ruff check and
ruff format --check clean.

Known limit (deferred)

Orphans of a stem the current run does not write -- a dropped input, or a collider
whose disambiguated stem changed between runs -- stay warn-only, since prep can't
tell them from data the user curated on purpose. A manifest would let --clean
reach those safely: filed as #84.

Refs #8
wake-20260723T1005-20d30f

…ips (#8)

Re-recording a take with fewer utterances and re-running prep into a populated
--output dir left the previous run's higher-numbered clips on disk. Training and
eval glob every clip in the directory, so those orphans were silently folded into
the next run's data while process() reported only the count it had just written.

prep already detected the leftovers and warned; --clean now removes them in the
same run so the reported count matches what is on disk. Deletion is scoped to the
current run's own stems -- a "<stem>_NNN.wav" clip whose stem this run wrote -- so
a source recording, an unrelated take's clips, or hand-curated audio in the
directory is named but never auto-deleted. A stray --output can't wipe files prep
did not create, and without --clean the warn-and-name behavior is unchanged.

The stem match is a greedy "(.+)_\d{3,}\.wav" so an orphan pairs back to its
disambiguated collider stem (computah-1_002.wav -> computah-1, not computah) and
the >999-segment case (:03d is a minimum width, so index 1000 is take_1000.wav)
still parses. Tests cover the re-run-with-fewer-clips case with and without
--clean, that --clean spares files this run did not record, and disambiguated-stem
orphan cleanup.

Orphans of a stem the current run does not write -- a dropped input, or a collider
whose disambiguated stem changed between runs -- stay warn-only, since prep can't
tell them from data the user curated on purpose. A manifest would let --clean reach
those safely; filed as #84.

Refs #8
wake-20260723T1005-20d30f

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 990795d801

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread prep_wake_samples.py Outdated
The cloud review caught a data-loss hole in the --clean stem-scoping: run_stems
was built from the input list (`stems`), so a take that produced zero clips this
run -- a silent or bad re-recording, or thresholds that dropped every segment --
still had its stem in run_stems, and --clean then deleted that stem's prior good
clips while writing no replacement. The narrow "only a re-recorded take's own
orphans" guarantee has to key off what the run actually wrote, not what it tried
to record.

Derive run_stems from `written` (each written path is a `<stem>_NNN.wav`, so its
stem reads back through _clip_stem). A take that wrote nothing contributes no
stem, so its earlier clips linger and warn instead of being wiped, and the only
copy stays safe.

test_clean_spares_prior_clips_when_rerun_writes_nothing reproduces it: a good
3-clip run, then a rerun whose --min-dur drops every segment; with --clean the
prior clips must survive. It fails on the old input-stem scoping and passes on
the written-stem scoping.

Refs #8
wake-20260723T1005-20d30f
@jamditis
jamditis merged commit 9c645ea into main Jul 23, 2026
1 check passed
@jamditis
jamditis deleted the fix/8-clean-stale-clips-on-rerun branch July 23, 2026 23:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant